QBCrowd: A Crowdsourced Question Bank for Exam Paper Generation

Abstract

The traditional method of setting a question paper by a particular person teaching in the university. This method is limited in terms of the number of questions, the difficulty of the question paper, and the experience of the lecturers. Generating large question banks can aid in generating a question paper with a random difficulty level. This can be useful for schools, colleges, autonomous Institutes, and Universities. We propose, designing a portal that allows the lecturers from various institutes to log in and submit the questions to the board and another login for the board or the administrators who can check all the questions and validate the questions so that the question paper or a question bank can be, generated automatically by the system. Lecturers can set the level of difficulty i.e., Easy, average, and difficult for each question while submitting the questions and board members can instruct the automatic question paper generator system to set the difficulty of question paper by allotting the number of questions to be picked from each difficulty type. The system will detect the questions that have the same semantics but a different representation to avoid ambiguity while generating the question paper.

Introduction

I. INTRODUCTION

For setting question papers, we propose a system, which will generate the question paper with little to no need for human interaction, which will hugely reduce the manual efforts of institutions. This system can be adopted by almost every institute which wishes to conduct examinations. This system could decrease the burden of gathering question banks and choosing the questions to set on the question paper. This is because question banks are generated by gathering different questions from different lecturers all over the country, this would create a huge database of questions and from this, completely random questions can be selected for generating question papers. With the help of randomization techniques, we can make sure that no questions are repeated more than once in the question paper, and this technique will also help in similarity checking, which is completely done automatically. Creating Crowd Sourcing Model for Generating Question Banks and Question Papers where we propose, designing a portal that allows the lecturers from various institutes to login and submits the questions to the board and another login for the board or the administrators who can check all the questions and validate the questions so that the question paper or a question bank can be, generated automatically by the system.

II. PROBLEM STATEMENT

To be able to graduate to objective-type questions for one semester of online university exams, a question bank of at least 5000 questions will be required for each subject. Setting question papers for exams is a complicated task. We have to think of a Crowd Sourcing model where questions are set by a large number of anonymous stakeholders thereby creating a large question bank. These questions can be vetted by experts before freezing the same in the question bank. The actual question paper can be set through an automated system. Our review of the relevant literature has revealed no prior studies or implementations of crowdsourcing in the process of writing and evaluating questions collectively by university instructors to develop exams.

Through this QBCrowd, A virtual crowdsourcing platform can help you in creating and evaluating questions for a preparatory course in a university setting by automation. With the help of this system, we provide an eco-friendly and easy solution to generate question papers for university exams from the ocean of questions created by the contribution of some brilliant teachers.

III. PROPOSED SOLUTION

QBCrowd is a role-based web application that provides a speci?c interface based on the users’ roles. For instance, it provides teachers with two interfaces one that allows them to add questions and the other to evaluate the questions. Furthermore, it provides administrators with basic statistics.

The task of developing exams for the course is often done by a small community of instructors who write the question items and send them for review by, usually one or two, senior instructors or ‘exam experts. The experts review and/or edit questions and then decide whether a question item merits inclusion in the exam. 

In recent review of crowdsourcing studies in education, mapped crowdsourcing projects and activities in education into four primary strategies:

A. Collective Intelligence or Crowd Wisdom

This refers to crowdsourcing projects or activities that seek knowledge sharing and ideas generated from the crowd to solve problems, predict future outcomes, and help educational institutions achieve their strategies.

B. Crowd creation

This refers to the process of involving crowd sources, e.g., students, staff, and faculties, in creating or co-creating a product or service.

???????C. Crowd Funding

The process of pooling or raising money from different parties, via the Internet, for the purpose of funding educational projects, student scholarship, or academic research projects.

???????D. Crowd Voting.

The process of soliciting public opinions, decisions, or ideas by the way of voting.

This task needs to be done by skilful exam developers which many institutions may be in shortage of. Moreover, based on our observation, exams developed by a small committee, especially when there is a time constraint, contain some defects such as poor-quality question stems and choices, and spelling mistakes. More importantly, this conventional method does not involve and consider the opinions of most instructors who teach the course and know their students’ abilities more than anyone else.

1. For Admin

a. Maintain the question bank.

b. Assign and change roles.

c. Has all access rights.

2. For Reviewer

a. Review submitted questions and give feedback on rejection.

b. Freeze spam user's accounts.

c. Contribute questions like any other contributor.

d. Request contributions in topics having a smaller number of questions.

3. For Exam-setter

a. Have all rights of a reviewer.

b. Generate question papers through an automatic system.

c. The permission to generate the paper will be valid for 48 hours and after that exam setter's role will get demoted to the reviewer by the admin.

d. View and download the previously generated question papers.

4. For Contributor

a. Contribute questions via a form.

b. Manage their pending, rejected, and approved questions (View/Delete).

c. View previous year's papers.

d. Be rewarded for contributions through the QBCrowd points system.

IV. QUESTION ITEMS CREATION AND EVALUATION

The System is divided into four parts namely Admin, Reviewer, Exam Setter and Contributor.

A. For Admin

1. Maintain the question bank.
2. Assign and change roles.
3. Has all access rights.

B. For Reviewer

1. Review submitted questions and give feedback on rejection.
2. Freeze spam user's accounts.
3. Contribute questions like any other contributor.
4. Request contributions in topics having a smaller number of questions.

C. For Exam-setter

1. Have all rights of a reviewer.
2. Generate question papers through an automatic system.
3. The permission to generate the paper will be valid for 48 hours and after that exam setter's role will get demoted to the reviewer by the admin.
4. View and download the previously generated question papers.

D. For Contributor

1. Contribute questions via a form.
2. Manage their pending, rejected, and approved questions (View/Delete).
3. View previous year's papers.
4. Be rewarded for contributions through the QBCrowd points system.

??????????????

Conclusion

This system aids in decreasing the cost, time and effort required to gather questions and generate question papers. In addition, it ensures that the source of questions in question banks is authentic and generates question papers avoiding security problems. In addition, we plan to implement the same system for online-based exams as well. Using crowdsourcing techniques in the development of exams can help in increasing the quantity and the quality of exam questions given a large number of contributors and evaluators. In this paper, we described the architecture of the QBCrowd platform, and we showed its potential signi?cance in developing high-quality exam questions. In our future studies, we will investigate empirically the e?ectiveness of QBCrowd in producing high-quality exam questions. We will also examine the impact of adding gaming elements on the engagement of QBCrowd users. Finally, we will evaluate the e?ectiveness of providing special training and tutorials on question writing on the overall quality of questions produced by QBCrowd users.

References

[1] E. Estelles-Arolas and F. Gonzalez-Ladron-de Guevara. Towards an integrated crowdsourcing de?nition. Journal of Information Science, 38(2):189–200, 2012. [2] B. Solemon, I. Ari?n, M. Din, and R. Anwar. A Review of the uses of Crowdsourcing in Higher Education. International Journal of Asian Social Science, 3(9):2066–2073, 2013. [3] L. de Alfaro and M. Shavlovsky. CrowdGrader: Crowdsourcing the Evaluation of Homework Assignments. In Proceedings of the 45th ACM technical symposium on Computer science education, number August, pages 415–420, 2014. [4] D. Brabham. Using Crowdsourcing in Government. IBM Center for The Business of Government, pages 1–42, 2013. [5] T.T.Hills Crowdsourcing content creation in the classroom. Journal of Computing in Higher Education, 27(1):47–67,2015.

Copyright

Copyright © 2022 Parag Soni, Rajil Jain, Shivansh Swarnkar, Yash Pal. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.